Papers with downstream benchmark tasks
exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train pre-trained models with limited corpus and computational resources are limited by the complexity of the training resources. |
| Approach: | They propose a method to extend BERT pre-trained models from a general domain to a new pre-train model for a specific domain with a different additive vocabulary. |
| Outcome: | The proposed method outperforms existing methods on biomedical benchmark tasks using the MTL-Bioinformatics-2016 dataset. |
Few-Shot Table Understanding: A Benchmark Dataset and Pre-Training Baseline (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models have demonstrated their effectiveness for few-shot table understanding, but few-shoot table understanding is rarely explored due to the deficiency of public table pre-training corpus and well-defined downstream benchmark tasks. |
| Approach: | They establish a benchmark dataset and use it to explore few-shot table understanding in Chinese. |
| Outcome: | The proposed model improves the few-shot table understanding in Chinese. |